Skip to content

fix: route llama.cpp log fallback to stderr instead of stdout#609

Open
andreinknv wants to merge 1 commit into
withcatai:masterfrom
andreinknv:fix/addon-log-route-to-stderr
Open

fix: route llama.cpp log fallback to stderr instead of stdout#609
andreinknv wants to merge 1 commit into
withcatai:masterfrom
andreinknv:fix/addon-log-route-to-stderr

Conversation

@andreinknv
Copy link
Copy Markdown

What

When the addon's llama.cpp log callback has no JS logger to delegate to —
either none was registered yet, or a registered logger threw — it falls
back to writing the log line directly. That fallback sent every
non-error line (warn / info / debug) to stdout; only error-level
lines went to stderr.

This routes all llama.cpp log output to stderr in both fallback sites
(addonCallJsLogCallback and addonLlamaCppLogCallback).

Why

For an embedder whose stdout is a data channel, the current behavior
corrupts output:

  • A CLI that prints results to stdout gets llama.cpp's load: …,
    init: …, llama_*: … model-load chatter interleaved into those
    results — dozens of init: embeddings required but some input tokens were not marked as outputs -> overriding lines per run for an
    embeddings workload.
  • An MCP / stdio server is worse: its stdout is a JSON-RPC stream, and
    non-JSON log lines on it are protocol corruption.

Library diagnostics belong on stderr regardless of level — that's the
standard contract, and what llama.cpp itself defaults to. The fallback
only fires when the JS logger is unavailable, so this changes nothing for
consumers with a working logger; it just makes the no-logger fallback
safe.

Change

Both fallback blocks previously did if (level == error) stderr; else stdout;. Since both branches now target stderr, I collapsed each to a
single fputs(…, stderr). If you'd prefer to keep the explicit level
check for readability, happy to restore it — the only intended behavior
change is non-error → stderr.

Testing

Verified with an embeddings workload that previously interleaved ~120
init: lines into stdout: stdout is now clean and the lines appear on
stderr.

Touches only llama/addon/globals/addonLog.cpp. No JS / API change.

When no JS logger is registered, or a registered logger throws, the
addon's log callback falls back to writing llama.cpp's output
directly. That fallback sent every non-error line (warn / info /
debug) to stdout.

For an embedder whose stdout is a data channel that is corruption:
the `load:` / `init:` / `llama_*` model-load chatter gets interleaved
into a CLI's results, or — worse — into an MCP / stdio server's
JSON-RPC stream. Library diagnostics belong on stderr regardless of
level.

Both fallback sites (addonCallJsLogCallback and
addonLlamaCppLogCallback) now write all llama.cpp log output to
stderr. Error-level lines already went there; this moves only the
non-error lines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant